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Knowing which mode of combinatorial regulation (typically, AND or OR logic operation) that a 
gene employs is important for determining its function in regulatory networks. Here, we introduce 
a dynamic cross-correlation function between the output of a gene and its upstream regulator con- 
centrations for signatures of combinatorial regulation in gene expression noise. We find that the 
correlation function is always upwards convex for the AND operation whereas downwards convex 
for the OR operation, whichever sources of noise (intrinsic or extrinsic or both). In turn, this fact 
implies a means for inferring regulatory synergies from available experimental data. The extensions 
and applications are discussed. 



PACS numbers: 87.18.-h, 05.45.Tp, 87.16.Yc 

Cells live in a complex environment and continuously 
have to make decisions for different signals that they 
sense. A challenge in systems biology is to understand 
how signals are integrated. As the central information- 
processing units of living cells, transcription regulatory 
networks allow them to integrate different signals and 
generate specific responses of genes. The elementary 
computations are performed at the cis-regulatory regions 
of the genes: the transcription rate of each gene (the out- 
put) is a function of the active concentrations of each of 
the input transcription factors (TFs) Such a quan- 
titative mapping between the regulator concentrations 
and the output of the regulated gene is known as the 
czs-regulatory input function (CRIF), which can be func- 
tioned as implementations of Boolean logic [2 [5] in anal- 
ogy to Boolean calculations that basic electronic devices 
perform [4J. For example, two activators regulate a gene 
with AND or OR logic operation (refer Fig. 1). The 
notion of logic operations can also be generalized by in- 
troducing a continuous function that encodes the depen- 
dence of the rate of transcription on the concentrations 
of inputs [T . Knowing which mode of combinatorial reg- 
ulation that a gene employs is important for determining 
its function in regulatory networks. For example, the 
cis-regulatory module drives cellular patterns differently 
depending on how the gene integrates intracellular and 
extracellular signals at its regulatory region by endoge- 
nous and exogenous TFs [5] [B] ■ 

Experiments performed on single cells have revealed 
that because TFs are often present in low copy num- 
bers, stochastic fluctuations or noise in the concentra- 
tions of these molecules can have significant influences on 
gene reg ulation [3 [3 HI HU] • The traditional fluctuation- 
dissipation relation derived by the linear noise approach 
[IT] based on the mater equation gives the informa- 
tion only about the second-order moments. Recently, a 



modified fiuctuation-dissipation relation was derived by 
Warmfiash and Dinner [12^, which relates some third- 
order moments evaluated at the system steady state to 
the derivatives of a CRIF. Such a static cross correlation 
provides the information only about how three time series 
are correlated at the zero correlation time. From view- 
points of gene regulation, however, the binding of TFs 
to the DNA is context dependent, active in some genetic 
states but not in others. In particular, stochastic fiuctua- 
tions, or 'noise', in gene expression propagate from active 
inputs to the outputs of regulated genes during signal 
integration. Thus, dynamic cross correlations |13l 114] 
would provide a noninvasive means to probe modes of 
combinatorial regulation in gene expression noise. The 
purpose of this Letter is to demonstrate its potentials 
in detecting signatures of combinatorial interaction. Re- 
garding the study of combinatorial regulation, there are 
other works [THl [HI EZl HH US]- Usually, these papers 
used some real time-course microarray data to test their 
algorithms and identify some synergistic TFs. 

Before presenting our analysis, let us examine a real 
biological example. Consider a genetic circuit based on 
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FIG. 1: (color). Schematic illustration of cis-regulatory con- 
structs. The regulatory functions are realized through the 
regulated recruitment of transcription factors and RNA poly- 
merase (RNAP). 
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FIG. 2; (color). Geometric characteristics of dynamic cross 
correlations for the phage-A operon, where 10"^ cells are mea- 
sured. There is an anti-correlation relationship between the 
convexity of dynamic cross-correlation curves for AND and 
OR operations. 



the phage-A operon [T5J|2n]- The corresponding biochem- 
ical reactions are Hsted in the Supporting Material [5T], 
wherein how intrinsic and extrinsic noise sources gener- 
ate are explained. We first perform realistic stochastic 
simulations of the whole circuit by using biologically rea- 
sonable parameter values and obtain three time series 
data of input TFs Si{t) and S2{t) and the output So{t) 
[25] . We expect these simulations to faithfully reflect the 
biological system because the phage-A is a well-studied 
system for which many parameters are measured and 
comparable models are capable of accurately reproducing 
distributions of protein concentrations in prokaryotic sys- 
tems ^23j|24j. Then, according to Ref. |2T], we calculate 
dynamic cross-correlation functions i?s^s2^s(,(r) for AND 
and OR operations, respectively. Figure 2 shows the 
dependence of the normalized dynamic cross-correlation 
function R{t) on the correlation time t. Apparently, the 
correlation curve near the peak point close to the zero 
correlation time is upwards convex for AND operation 
and downwards convex for OR operation, whichever the 
sources of noise (intrinsic or extrinsic noise). 

Such an anti-correlation relationship between the con- 
vexity of dynamic cross-correlation functions for AND 
and OR operations is not a casual finding but is a gen- 
eral fact. In what follows, we will analytically verify this 
point using a simple yet general model as schematized 
in Fig. 1. The corresponding biochemical processes are 
modeled with the production and degradation of the TFs 
and the output only 
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(1) 



where Si and S2, both of which are activators, repre- 
sent the TF inputs to czs-regulatory module, So is the 
measured output of the regulated gene, and arrows from 
and to denote synthesis and degradation, respectively. 
The production rate of So is determined by the concen- 
trations of the TFs and is encoded in the (dimensionless) 
ds-regulatory input function CRIF(S'i, S'2) (see Ref. [5T] 
for its analytic form). 

Note that the accurate modeling of the system (1) 
should adopt the master equation but to show our 
analytic results, we instead take the following simplified 
Langevin equations 



dSi 
~dt 
dS2 

dt 
dSp 

dt 



ai+E + h- fiiSi 
a2 + E + I2- (^282 
CRlF{Si,S2) + E + Io~poSo. 



(2) 



Such an approximation can still describe well the mo- 
tion of individual species molecules under some ideal 
conditions (see Ref. [21] for interpretations). The 
above equations include terms for protein production rate 
(ofi, i = 0,1,2), protein degradation and dilution rate 
(/?i, i = 0, 1, 2), and the contributions of intrinsic and ex- 
trinsic noise sources (/^ {i — 0, 1,2) and E respectively). 
Here, the extrinsic noise E is defined as a stochastic fluc- 
tuation to globally measured components, whereas the 
intrinsic noise is assumed as stochastic fluctuations in 
the gene expression. Noise sources are modeled using 
Ornstein-Uhlenbeck processes by 



dE 
~dt 
dh 
dt 



= ~(5eE + aETjE 

= -kJi + ai-qi {i = 0, 1, 2). 



(3) 



Assume that the white noise terms 77^, 771, 772 and ryo 
are independent, identically distributed processes with 
the zero mean and the unit standard deviation. The 
parameters (3i and Ki define the time scale of the noise, 
while ge and Ui {i = 0, 1, 2) set the standard deviation. 

We expect perturbations due to noise to be so small 
that it might be valid to approximate our system using 
the second-order Taylor expansion of CRIF at the origin 
system. Denote S'^'^ = ai/[3i {i = 1,2). Define Si ~ 



S, ~ 5f(i = 0,1,2), where S^" = CRlF{Sl\ S^") 



with ao 



511 



((s?))t+5l2((siS2»t- 



Oo 



((52))* in which 



522 
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the outside bracket represents the average over the time 
t, and 5ii, 512, 522 are 2-order derivatives of the function 
CRIF with respect to variables ^i and 5*2, evaluated at 
the point (S"!*^, Sj'). This will result in the following 
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equations 

^ = E + h-/3,s, 
at 

^ = E+l2-^2S2 (4) 

^ = E + Iq- PaSo + gisi + 9232 
at 

, 5ll 2 , , 522 2 

For simplicity, we assume P — Pi and k — Ki {i — 0, 1, 2), 
and without loss of generality, also assume P k, P 2k 
in the following analysis. By calculation, we find oq = 



{gii+2gi2+922)^l/{8p^) + {gii<yl+g22<y^)/{4Pi^iP+i^)). 

Finally, define the dynamic cross correlation between 
So{t) and Si(t), S2(^) as 

Rs^s,,so{r) = {{si{t)s2{t)so{t+T)))t, (5) 

where r represents the correlation time. In sim- 
ulations, this function is normalized to R{t) = 
Rs^s2^so{T)/y^RsiS2,stS2{0)Rs„,so{0)- By complex calcu- 
lations, we obtain the analytic expression of the unnor- 
malized dynamic cross-correlation function |21j . denoted 

by i?int(T), 



3/33 /3 2k /3(2/3 -t- k) 



(a) o-ot 
0.02 

-0.02 

-0.04 



ANOCie'O^OS) 




OR(<Te=0) 

OR(<1e=0.005] ' 





(b) 



„ 

E 



c -2 



-400 -200 



200 400 




FIG. 3: (color). Description of dynamic cross correla- 
tions in the modeled system, (a) The geometric character- 
istics of R{t), where K = lOOnM, n = 2, ai = 02 = 
Imolecule/cell/min, ao = 4molecules/cell/min (a parame- 
ter in the CRIF function. See Ref. [2T]), /3 = 0.01/min, 
K = 0.02/min, ai = (T2 = (Jq = 0.02(molecules/cell)^''^/min. 
The empty circles represent simulated results whereas the 
symbols indicated in the figure represent theoretical results; 
(b) The dependence of the 2-order derivative of the correlation 
function R(t) at the peak point r™, R"{r,n), on the noise in- 
tensity a and the signal concentration S, where cti = a2 ~ f, 
(To = f/10, Si — S2 ~ S and the other parameters are similar 
to those in (a). S20 and Sgo represent 20% and 80% maximal 
values of the input signal concentrations, respectively. 



in the presence of intrinsic noise only, where 7 = 

Note that the sign of 9^2 
is opposite for the AND and OR operations (see Ref. 
[21]'). The simple analysis shows that i?int(''') has one 
peak at some small > 0. In particular, the convexity 
of i?int (t) at a small interval of > but close to t = 
is anti-correlative for the two logic operations, referring 
Fig. 3(a). 



In the simultaneous presence of extrinsic and intrin- 
sic noise, the total unnormalized cross-correlation func- 
tion can be expressed in the form of R{t) = Rint{T) + 
^cxt(''') + i?mix (■'■), where i?oxt(''') represents the dynamic 
cross correlation in the case of extrinsic noise only and 
^mix(''') represents the cross terms due to the coopera- 
tive effect of intrinsic and extrinsic noise. The analytic 
expressions of -RGxt(''') and -Rmix(''") are put in Ref. [21"]. 
Figure 3(a) shows that the extrinsic noise does not in- 
fluence the convexity of the correlation function R{t) for 
both logic operations, where the theoretical results are in 
good accord with the numerical results. Note that there 
is a difference in the effect of extrinsic noise on the loca- 
tion of the dynamic cross-correlation curve between Figs. 
2 and 3(a) in the case of OR operation. That is, extrin- 
sic noise uplifts the dynamic cross-correlation curve in 
Fig. 2, but it moves down the dynamic cross-correlation 
curve in Fig. 3(a). This is possibly because for the mod- 
eled system, the additive noise of capturing the effect of 
external fluctuations does not depend on the state vari- 
ables whereas for the real system, the extrinsic noise that 
appears actually in the relevant Langevin equation is de- 
pendent of the state variables [25 . Figure 3(b) further 
shows that the convexity of i?(r) is robust to noise in 
the active region of the two input signals (here, by the 
active region we mean that concentrations of the input 
signals are beyond 20% of their maximal values [26 ). 
This is because the 2-order derivative of R{t) evaluated 
at the peak point, denoted by R"{Tm), the sign of which 
describes the local convexity of R{t), is always negative 
(i.e., upwards convex) for the AND operation whereas 
positive (i.e., downwards convex) for the OR operation 
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in this active region. 

In conclusion, we have shown that the dynamic cross- 
correlation functions for AND and OR operations in gene 
expression noise have apparently distinct geometric char- 
acteristics (convexity). Such a difference is qualitative, 
depending neither on specific models nor on the sources 
of noise, and hence the essential difference reflected by 
the modes of combinatorial regulation. Moreover, since 
the dynamic correlation function utilizes statistics of the 
naturally arising fluctuations in the copy number of the 
species, its geometric characteristics can in turn help 
us efficiently detect signatures of combinatorial regula- 
tion with available experimental data. This is useful be- 
cause proximity in DNA binding is not sufficient to infer 
combinatorial interactions, and they cannot be readily 
probed by traditional methods (e.g., knockouts) or high- 
throughput expression assays (e.g., microarray data). 

Since stochastic fluctuations, or noise, exist inherently 
in biochemical reactions, using noise rather than exter- 
nal interference means to mine bioinformation related to 
gene regulation provides a new research line. Regard- 
ing this aspect, there have been some works, e.g.. Cox 
et al. used noise to characterize some genetic circuits 
[?f] . Dunlop et al. used correlation in gene expression 
noise to reveal the activity states of regulatory links [H] . 
Warmflash and Dinner used static cross correlations to 
detect signatures of combinatorial regulation in intrinsic 
biological noise [T5]. We utilized dynamic cross correla- 
tions based on the nature of noise correlation to iden- 
tify the modes of combinatorial regulation in intrinsic or 
extrinsic noise or both. In contrast to Warmflash and 
Dinner's approach, our approach would have some ad- 
vantages since dynamic cross correlations can in general 
provide more information about gene-gene correlation in 
expression than static cross correlations. 

The method of dynamic cross correlation can also be 
extended to other situations of logic operations (ANDN, 
ORN, NAND, NOR). For example, consider a system 
with two input TFs and the output of a gene. If both 
TFs are activators, this case has been studied in this pa- 
per; If both are repressors, our method can still show 
that the dynamic correlation function i?(r) is upwards 
convex for NOR whereas downwards convex for NAND; 
If one TF is activator and the other is repressor, the 
R{t) is upwards convex for ANDN whereas downwards 
convex for ORN. In the cases of XOR and EQU, how- 
ever, the approach will be invalid since the input TFs 
may be activator or repressor. Except for inferring syn- 
ergies between regulators, the idea of dynamic correla- 
tion (e.g., 2-point dynamic cross correlations introduced 
in Ref. [T11[2S]) can even be used to determine the direc- 
tion and relationship of interactions between arbitrary 
two regulators, i.e., to determine who regulates whom 
and who activates/represses whom. The details will be 
discussed elsewhere. Finally, the approach of dynamic 
cross correlation can be applied to other biological net- 



works, e.g., RNA logic devices nucleic acid logic cir- 
cuits |30], signaling protein logic modules [3T|, to identify 
the types of logic operations. 

This work was supported by the Natural Science 
Key Foundation of People's Republic of China (No. 
60736028). 
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The Supporting Information includes the following contents: (1) list biochemical reactions and 
parameter values used in the main text for the system of the genetic circuit; (2) simply state the 
numerical method of computing cross-correlation fiinction in the discrete case; (3) give the 
expression of cw-regulatory input function (CRIF); (4) derive the analytic expressions of dynamic 
cross-correlation functions; and (5) compute normalization factor. 

1, Biochemical reactions and parameter values for genetic logic gates 

A number of studies have shown that gene networks have cw-regulatory elements governed by 
Boolean- like logic [1-22]. We consider a genetic logic gate based on the phage-^ operon [1,14]. In 
the construct of this system, the Prm promoter and Or2 binding site are in their natural locations 
and an additional binding site for the Escherichia coli lac activator CRP is located upstream; the 
cl activates transcription by binding to Or2 and the output is lacZ. The original construct is an 
AND logic gate. Similarly, we can also construct an OR logic gate by a few point mutations 
[15,16]. 

Denote by D;(! =0,1,2) DNA regulatory sequences of genes, which encode proteins lacZ, 
CRP and cl (S,(/ = 0,1,2)). Also, denote by M,(; =0,l,2)the mRNA molecules and P the RNA 

polymerase. The genes can randomly produce and degradate the proteins with the same rate, but 
the production or degradation depends on the state of the operator Dj(j = 0,1,2) . Note that in our 
case, transcription factors (TFs) Sj and S2 are taken as inputs whereas as the output. The 
cw-regulatory constructs are shovra in Fig. 1 in the main text. Assume that TFs S, and can 



combinatorially bind to the operator in the form of monomer, hi the case of AND gate, the 
output gene can be expressed only when both input TFs S, and bind to the operator 
co-operatively. In the case of OR gate, however, the output gene can be transcribed when one or 
both of the input TFs bind to the target operator. The parameters listed in Table SI for the OR 
logic gate are from Ref. [14], which are used to simulate the idealized logic gates. The parameters 
for the AND logic gate are the same as for the OR logic gate except that the transcription rates are 
set to zero when only one input TP binds to D„ . 

In order to illustrate sources of noise, such as the presence of intrinsic noise only and the 
simultaneous presence of extrinsic and intrinsic noise, we give the corresponding chemical 
reactions separately in Tables SI and S2, which are divided into two categories: reversible 
(DNA-binding reactions and multimerization) and irreversible (transcription, translation and 
degradation). In the idealized logic gate (Table SI), we assume that the rate of the DNA state 
change is fast enough and the fluctuating DNA state has been replaced with the equilibrated state 
by neglecting the explicit dynamics of DNA state alteration (which is called the adiabatic 
approximation) [23]. In this strong adiabatic limit, the stochastic fluctuations in three genes 
involved in logic gates lead to so-called intrinsic noise. Since the DNA state alters much more 
slowly in eukaryotes than in prokaryotes, the actual dynamics of the DNA state can be in the 
weakly adiabatic or nonadiabatic situation, which can be modeled by transitions between "on" and 
"off" states. In addition, genes in a single cell may be affected by some global fluctuations (i.e., 
so-called extrinsic noise) [24], such as fluctuations in the number of RNA polymerase molecules 
or ribosomes, and variations in cell sizes. To explore the effect of extrinsic noise in more natural 
setting, we explicitly include the detailed processes, such as the DNA-binding proteins to recruit 
RNA polymerase and DNA state, in an extended set of chemical reactions (see Table S2). 



TABLE SI: Reactions and parameter values for simulations of the logic gate in the idealized 
system (used in the presence of intrinsic noise only). The reactions include the TF binding to the 
DNA promoter, transcription, translation, and degradation of mRNAs and proteins. 



Descriptions 


Reactions 


kf 


K 


Transcription 


0^Mi 


5 




Translation 


Ml ^Mi+Si 


10 




mRNA degradation 


Ml ^0 


1 




Protein degradation 


Si ^0 


1 




Transcription 


0^Mj 


5 




Translation 


M2 ^ M2 + S2 


10 




mRNA degradation 


M, ^0 


1 




Protein degradation 


S^ ^0 


1 




RNAP binding to DNA promoter 


Do+Si^D„Si 


10 


500 


RNAP binding to DNA promoter 




10 


500 


S2 binding to DqSi complex 


D„Si+S2^D„S,S2 


10 


250 


Si binding to D0S2 complex 


D„S,+S, ^D^SiS^ 


10 


250 


Transcription 


Do ^-Do+Mo 







Transcription 


D„Si^D„Si+Mo 


20 




Transcription 




20 




Transcription 


D„S,S2 ^D„SiS2+M(, 


20 




Translation 


M„ ^M(,+So 


10 




mRNA degradation 


M(, ^0 


1 




Protein degradation 


So^0 


1 





TABLE S2: Reactions and parameter values for simulations of the logic gate in the more real 
system (used in the simultaneous presence of extrinsic and intrinsic noise). The reactions also 
include the TF and RNA polymerase binding to the DNA promoter, transcription, translation, 
degradation of mRNAs and proteins, and DNA-binding proteins to recruit RNA polymerase and 
DNA state. 



Descriptions 


Reactions 




K 


RNAP binding to DNA promoter 


Di+P;F^DiP 


10 


500 


Transcription 


DjP^DjP + Mi 


5 




Translation 


Ml ^M, +Si 


10 




mRNA degradation 


M, ^0 


1.3 




Protein degradation 


Si ^0 


1 




RNAP binding to DNA promoter 




10 


500 


Transcription 


DjP^D^P + Mj 


5 




Translation 


^Mj+Sj 


10 




mRNA degradation 


Mj ^0 


1.3 




Protein degradation 


S2 ^0 


1 




RNAP binding to DNA promoter 




10 


960 


Si binding to DqP complex 


D„P + Si^D„PSi 


10 


500 


S2 binding to DqP complex 


D„P + S,^D„PS, 


10 


500 


S2 binding to DqPSi complex 


D^PSi+Sj ^F^DjPSiS^ 


10 


250 


Si binding to D0PS2 complex 


DoPSj+S, tF^DjPSiSj 


10 


250 


Transcription 


D„P^D„P + M„ 







Transcription 


D(,SiP^D(,SiP + M(, 


20 




Transcription 


DoSjP^DdS^P + Mj 


20 




Transcription 


D0PS1S2 ^DgPSiSj+Mo 


20 




Translation 


M„ ^M„+So 


10 




mRNA degradation 


M„ ^0 


1.3 




Protein degradation 


So^0 


1 





2. Computation of cross correlation functions in the discrete case 

Given single-cell time series data of input transcription factors 5i(0 and SjCO and the 
output Sfiit) , we compute the difference between the value of 5,(0 and its average: 

Sj(t) = Sf(t)-(^S;(t)) , where j = 0,l,2 , and the 3-point dynamic cross-correlation function 
R, , , (r) according to the formula 




(SI) 



1 " 

where s. = S. ^ S,. (n) , and N is the number of point series. This fimction is normalized to 




3. CIS-regulatory input function (CRIF) 

Consider the system of two activators. There are four binding states of the promoter D: D, DSi, 
DS2, DS1S2, coiTesponding to each combination of bound transcription factors. If the binding and 
unbinding of transcription factors to the DNA sites are taken to be fast, ci\s-regulatory input 
function (CRIF)[14-22] can be set as 



CRIF(5',,5'2) = a„- 



(S3) 



1 + {SJKJ+{SJKJ+{SJKJ{SJKJ 
where «„ describes the dimensionless transcription rate, K- (i = 1, 2) is the (equihbrium) 
dissociation constant for the binding of the transcription factor 5, (/ = 1,2) , n is Hill coefficient, 
which describes the cooperativity. For AND gate, both regulators must be bound to initiate 
transcription so = r, = = and r^^=l , whereas for OR gate, binding of either regulator 
enables the maximal production so that r„ =0 and = r^= r,^ = 1 .The mixed-petrtial divertive is 



dS.dS, 



(S4) 



S,S,(l + {SjKj) (l + {SjKj) 
The following figure shows the dependence of the second-order mixed partial derivatives ( g^2 ) on 
the input signal concentrations. This figure can help us find the active region mentioned in the 
main text. 




% -2 -2 \oOk^"^ ^ 'V -2 -2 vo?t^ 

Fig. SI. The second-order mixed partial derivatives ( g,2 ) with respect to S, and Sj ■ Parameter 
values are «„=!, n = 2, K = K^=K2 =100. 



4. Deriving analytic expressions of dynamic correlation functions 

Before presenting analytic results, we make some explanations for sources of noise that will 
appear in our model. In general, noise in the form of random fluctuations arises in a biological 
system in one of two ways. As discussed in section 1 of this Supporting Information, internal noise 
is inherent in biochemical reactions. Its magnitude is inversely proportional to the system size, and 
its origin is often thermal. In contrast, external noise originates in the random variation of one or 
more of the externally set control parameters, such as the rate constants associated with a given set 
of biochemical reactions. If the external noise source is small enough, its effect can often be 
incorporated post hoc into the rate equations [25]. 

Assume that signals 5; , ^2 and obey the following kinetic equations: 

-^ = a,+E + I,-fiS^ 
dt 

^ = a,+E + I,-^S, (S5) 
dt 

^ = E + 1,-J3S„+CRIF{S„S,) 

In these equations, a. and /3 may be viewed as protein production rate and protein degradation 
and dilution rate, respectively. /, and E represent the contributions of intrinsic and extrinsic 
noise sources respectively, where the extrinsic noise E is defined as a stochastic fluctuation to 
globally measured genes, whereas the intrinsic noise is assumed as stochastic fluctuations in the 
gene expressions. Noise sources are modeled using Omstein-Uhlenbeck processes by 

— = -^£ + C7£% 

f (S6) 

We assume that the white noise terms t/^, t]^ and t]„ are independent, identically 
distributed processes with the zero mean and the unit standard deviation. The parameter and 
ic define the time scale of the noise (we always assume and P*1k in what follows), 

while and a. set the standard deviation. 

OC- 

Denoting S'^ = , «' = 1, 2 , we expect perturbations due to noise to be small such that it is 
valid to approximate our system using the second-order Taylor expansion of CRIF at the point 
(s;' , S'^ ) . In this case, we have: 



dCRIF dCRIF d^CRIF d^CRIF ^ d^CRIF . , „ 
where g, = , g, = , g., = — , g,, = , and g,, = r — with all 

functions gs evaluated at the point (S','''',^^'') . Defining s-^S--S"' (( = 0,1,2), where 

S;" =CRIF{S"',Sl'') + a„ with a„ =^^(.s,')^ +g,, ((^,5,))^ +^^(5,')^ in which the outside 
bracket represents the average over the time t , we have the following dynamical equations: 

^ = E + I,-p,s, 
at 

^ = E + ^-P,s, (S7) 
at 

ds. 



ai II 

The cross-correlation function between s^{t) and s^{t) , s^it) is defined as: 

R(T)={{s,(t)s,(t)s,(t + T)))^ (S8) 

where (•)^ represents the average over the time t . It follows from Eqs. (S7) that 

s. (f ) = s, (0)6-"' + £ e-^^'-'' dt^ + £ e"^''"'''/, {t, ) dt^ ,i = l,2 
s„ it) = s, (0).-^' - a„ II e-^^'-" ^dt, + II .-^<'-" '£ (f, ) dt, + £ e-^*'-" '/„ (f, ) dt, 
+g,£'.-^<'-"'., {t,)dt, ++g^_fy('-'^s, {t,)dt, 

+ S^l'y^'-"^sf {t,)dt, + g,j; {t,)s, {t,)dt, + S^iy^'-"\,l {t,)dt, 
Our assumptions to noise imply 

{{s, it)s, (t)s, (t + r)))^ = l^^s, (t)s, (t) -j + Y r ) 

+g4f (0^, (O'^f, +^r^"'*"^"''''^' (0^'.)) 

= iim|-^(.,(0.,(0)+^-^''^^'r«''''[^(^,(fK(0.v(O) 

+ 8n (^1 (0^2 (0^1 (fi ) ('1 )) + ^ (^1 (0^2 (0^2 ('1 )) dt, 

Since the cross correlation defined above does not depend on initial conditions, we may set 
>s,(0) = 0, i=0,l,2 (in other words, the initial values do not affect the resulting value of the 
dynamic cross correlation). In this case, we have 








According to our assumptions to noise, we further have 



+ {E{t,)E{t,)l,{u)l,{t,)) + 2{E{t,)E{t,)l,{t,)l,{t,))] dt,dt,dt,dt, 

(.,(o..(o^.^(o)=«"'*'"''nn^''''^"'"'^''*[(£(o^(0£(0£(o) 



+(£(?,)£(?3)7,(r,)/,(fJ) + 2(£(?,)£(r,)/,(f3)/,(fJ)]rff,^fr3^/r,^ff, 

(.,(o.ao^i(o*ao)=^"'*'"''nl}^'*''''"'''''[(^(^o^(0£(^4)£(o) 



+ {E{t,)E{t,)l, {t,)l, (f, )) + {E{t,)E{t,)l,{t,) I, {t, )) 
+ (A (^2)^ (^4)^2 (^3)^2 {h))]dt,dt,dt,dt, 

Collecting these expressions, we can express the cross correlation as 

R{t) = lime-^("'' f e^'' Y A, U )dt, -a 
j=i 

where 

a = ^\im{s,it)s,it)) = ^hme-"" \ (E(tAE{t,))dt,dt, 

^ gn+2gi2 + g22 . lij^ ^-2/., 1 1 ^M, -^)(^E{t,)E{t,)) dt,dt, 
2 

+i.lim.-^'f|/('--^>[g,(A(0A(r,)) + g,,(/,(0^2(f2))]rffA 
^ 00 

^ 0000 

A(0 = («n+gO^*''ii}^'*'^^''"'^'''(^(f2)£(f4)A(OA(0)^f2^M^4rf^S 


A3 (0 = {gn + g,,)e-^^(-"' J {E{h)E{u)l, {t,)l, {t,))dt,dt,dt,dt. 





Note that calculating the higher-order average of the noise E can be concluded as calculating its 
2-order average, thus yielding that 

{E{t,)E{t,)E{t,)E{t,)) = {E{t,)E{t,)){E{t,)E{t,)) 

+ {E{t,)E{u)){E{t,)E{t,)) + {E{t,)E{t,)){E{t,)E{t,)) (SIO) 

4 

_ r^-/'(|'2-'3| + |»4-'5|) g-fi(\h-k\ + \h-h\) _^g-/9(|«2-»3| + |«3-«4|)1 

4^' L J 
In addition, using the assumptions to noise we can have 

{e{t,)E{t,)L{t,) L {t,)) = {e{t,)E{t,)) (/, ) /, (r, )) 

rr' , , n-' , , ,« = 1,2 (S12) 

IP Ik 



(a {t,)l, {t,)l, (f,)) = (a {U)){^2 {h)h ih)) 



The substitution of Eqs. (S10)-(S13) into the expressions of A^-A^, a and and further 
R{t) yields 

f+r 

= lim e"''*"'' j e"'- [Sj (fj ) + (?i ) + 83 (fj ) + {t^ )] df, - a (S 14) 

where 

gi(0= ^"^g^f ^" ^^[-P'i(m)/^.(^.,0 + 2F/(m.)] (S15) 

^3(0=^^4ir^''^'^'^'''^^'^'"''^ (S17) 



a = — Vlim 



with 









The computation of the above 4 basic integrals Fj ~ is as follows. 



le""^-'"^dt,+\e'"^dt, 



-2/Jr 



where 







=e 



2^ l[l.2^(.-0]^^^"^^3«^^^e-^'-' 
Similarly, » ^^^^^^ — ■pjjg combination of both gives 



F,(f,r,) = 



l + y9(f-0 



-/»(>->■) 



< f, < f 



F3(f,f) = e-''"|e^'^^;f, 



P+K P-K p-K 



dt^ « 



P{P^k) 



(S19) 

(S20) 
(S21) 
(S22) 
(S23) 



where 



< f, < r 



7;3=e-^('"''|e^'=d?3 



55g--S(<+'l) 



Similarly, 



P+K P-K 



P-K 



. Thus, 



1 



.^e-A'-'O+e-C-'.) 
P 

P 



< f , < f 



The above means that the expression does not influence the resulting value of ^(r)) . 
Therefore, we have 



Zip' 



?11 '^812 822 _4 , 



1 

_1 
4p* 



+ 2 



- + 2 



16/c(y0 + /f)y9' 

p{t-t,)+l ^ 

2p' 

p{t,-t)+l 



■lp(t-h) 



0<t,<t 



2p^ 



(S24) 



(S25) 



[{8n + 812 ) + ( gi2 + ^22 ) c^2 ] 



4Pk 



l+^(f,-/) 



P 



0<ti<t 



(S26) 



2p\p^-K^) 

[8u<yl + 822^\](^\ 

\6k{P + k)P' 



P 



-e"-"^''-' +e- 



Ak'[P'-k') 



P 



e ' ' e 

P 



0<ti<t 



'('.-«) 



(S27) 



(S28) 



Case 1: In the presence of extrinsic noise only 

As r > , we have 



I+T I I [+T 



Furthermore, we have 



l^P 



+2 



10" 



Vf, +2 J 



%0 



2 CTfi lim 



2^9" 27;0 2^5" 



■ 1 



Therefore, 



152 
27 



As -r < r < (where T is an arbitrary positive constant), which corresponds to < fj < f , we 
have 

t + T 



=Hme 



Sn 2^12 + S22 _4 

5^ ^E- 



S0 



/'('-',) + ! 



-2fil+3fil 



ip^ 

\[p{t-t,) + qe"^'^dt. 



''dt, 



Or 



8n +^8,2+822 ,4 



8;ff' 



(Tj. lim 



Cj4 ' 



e - - n-24/3T + 9/3V 3Ar..) 



27;ff 



+ 2g,2 + ^22 ( 1 7 - 24/?r + 9P't' ] , 



\6P' 



21 



Combining both cases, we obtain 



Sn + 2^12 + §22 



511 ' 2^,2 + §22 



152 
27 



17-24/7r + 9;gV 
27 ^ 



r>0 



r<0 



(S29) 



Case 2: In the presence of intrinsic noise only 

As r > we have 

;+r ^ t t+T 



_ 812(^1 -fir |. 



e '■■ lim< [e 



dt, 



'('-'.) 



By calculation, we obtain the expression of R.^ (r) as t>0 



:re-^''' + e— +-e 

0" p-lK P 

For -r < r < , we have 

t+z 

"Him f 



P + K 



P-Ak^ P{2P + k) p" 



2kz ±_ -(P+k)t 



2 2 
8l2'^l(^2 



-/*('-',) 



4k'{P'-kJ 



dt, 



8l2<^l<yl ^-fi, i; 



lim f 



dt. 



Or 

/?. (r) = 



ip" P + 2k P{2P + k) 



Combining both, we obtain 



P' p-lK p 



K 



1 



2k 



3;0' P + 2k P{2P + k) 



(S30) 



with a,= ^'^"^'"^^ , and y = -A 

Ak\P'-k')' 



K P + K K 

P^-Ak^ P{2P + k) Jp" 



Case 3: In the simultaneous presence of intrinsic and extrinsic noise 

As r > , we have 





f I t + T 

= lime-'"-'*''>\le'"'B,{t,)dt, + J e"' B,{t,)dt, 
Vo < 

=a,-lime-''*'")|j[l + ^(f-r,)]e^'- 

t + T 



^g-2/*(»-<,) +g-(-8+'^)('-<.) 

/5 



iff, 



:^g-2A'.-')_,_g-(-9+«^)(<.-') 
/9 



where = 



[(gu + gi2 ) q-f + ( gi2 + g22 ) ] q"i 



. By computation, we obtain 



{2p + Ky 
, k{2 + I3t) P + k{\ + Pt) 

For -r < r < , we have 

t + T 



t + T 

=a,-\ime-''^'''^[\ + P{t-t,)y'' 





>9 



{2p + K) 

The combination of both gives 

"iP+K p + K _22k 
{2p + K)^~i? W 



5(4-3^ 2^r , 3yg + y _.(^,,), 



r>0 



r<0 



In addition, we have 



/^^,(r)^lime-^<-)je'*"53(0rf^,-a. 





(S31) 



(S32) 

\(>k{P + k)P'' 

Summarizing the above analysis, we finally obtain the expression of the dynamic cross-correlation 



function in the simultaneous presence of intrinsic and extrinsic noise: 
where 

'^152 



21 



r>0 



27 



P-lK P 



with a, = ^i^— ! — - — r- and r = — ^ 



3;ff'(;5'-4K-')(2;» + /c) 



1P + K p + K 22k 
{2P + k)'^~i? W 



'^{2 + Pt) _ P + K{\ + pr) 



with a, = 



^P' {2P + Kf 

[( gl 1 + gl2 ) + ( gl2 + g22 ) ] 
iK[p^-K^)p^ 



T>0 



r<0 



5. Computing the normalization factor 

Now, we calculate ^,,,^,,,,^(0) and /?,^ (0) . Note that 

Here we calculate the normalization factor N = xji^ ■yjRs.si .su^ (0) -sj^s, (0) ) • Note that 



(0) = ((.,(0^2(0^.(0^2(0)), = lim.-^' J J J J/('-«^«'"^) 



[{E{t,)E{t,)E{t,)E{t,)) + (A (0 A (^3 ) ^2 (f2 )/2 ih )) 

+ (£(f,)£(f3)A(f2)A(f4)) + (£(fl)£(f3)^2(f2)^2(f4))]^'fA^f3'^f4 

Using the previous calculation results, we can obtain respectively 

}i^^"'ini^''''"""^^(o^(O£(o^(o)=i#iis^.^(^'0='''^ 











Therefore, we have 

3o-£ o-'o-' +0-2) 

^..,..(0) = T^ + 4^v'(,9V;^)^^8^(/? + '^);»^ ^^^^^ 

In addition, we can express 

(0) = ((^o(0*o(0)), = q + C, + Co (S39) 

where 



C, = lim e-'^' \ \ {E{t, )E{t, ))dt,dt, + \ \ (/„ (r, ) ^0 (^2 )) dt.dt, 
[00 00 

+g,ff/('--''(£(0*,(f2))^fA+«2fJe''''-^'^'(£(0*2(^2))A^^2 



+gf f f {h)s, {h))dt,dt, + g,^f f {t,)s, {t,))dt,dt, 





+ ^\\e'^-^-^\sl{t,)4{t,))dt^^^^ 





r t 



+ ^ffe'*""'^(*f(0*2(f2))^A^f2+«nft2jJ^'^'-"^^(*,M0*>(0^2(0)^A^^2 



+ g,2g22ffe'*""^' (*2 (0*1 (^2)^2 {h))dt,dt. 



in which is given above. In what follows, we compute C, , and , respectively. 
Using the above results, we have 

lim.-^jj/('--)(£(0£(O)^A^r.=^limF,(M) = ^ 



lime-'^' j j/C'*'^) (f,))^Zf,rff, 



=lime-^^'jj/<''"^» e-^'^je'"^ {E{t,)E{t,))dt, 



dtidt2 



where e"^'' Je^'' (£(fi)£(r3))jr3 = 



Therefore, we can obtain 



lime-^'||/("^'^>(£(r,)^,(f.))^/rA =-^, where r=l,2. 



lim.-^' jf /<'-^) {s, {t,)s, {t,))dt,dt, = glime-^'f f /("^^'f, {t„t,)dt,dt, = 
lime-^^' j j/C'^'^) (0*, it,))dt,dt. 



9ai 



=lime 



-2/"JJg/'('l+'2) 



where i = 1,2. Thus, 



AP J IPk^P + k) AK{p + Kfp' 



(45) 



For Cj, note that 





+ (f3 ) A (^4 {h)h{h))+{E{h)E{t,)i,{t,)i, {t, )) 

+{E{t,)E{t,)l, {t,)l^ {t,))]dt,dt,dt,dt, 





+2{E{t,)E{t,)l, {t,)l, {t,))]dt,dt,dt,dt, 



+ {Hh )E{t,)l, {t,)lXh)) + 2{E{t,)E{t,) /, (f, ))] dt,dt,dt,dt, 



+(A {ts)l2{u)h {t,)) + {E{t,)E{t,)l, (t,)) 
+{E{t,)E{t,)l^{t,)l,{t,))'\ dt,dt,dt,dt. 

Using the above calculation results, we have 



)) 



,-2-8('i+'2) 



t, I, ij ij 



f,) + 2F,(f„f,)] 



,-2/'('2-'l) 



.-''('.-)))))/('3«-''«^)(/.(f3)/,(t,)/,(fJ/,(r,)) = ^F3(r,,OF3(f,,^ 



2 2 



,-2/'('.+<2)jj||/('3 



^D3(f,,f,) = 







^2 _2 
C^lC^2 



; A ( ^3 ) A ( ^5 ) ^2 ( ^4 ) ^2 ( ^6 ) ) = ^ ( A , ^2 ) ^4 ( , ^2 ) 



4k-- 



) _'^g-P(h-h) 



0<f, < 



g-'f(<2-'l) _^g-/'('2-'l 



g-2#,«2)||||/('3-. 





^2^2 
4y0A- 



E{t,)E{t,)L{u) L {(,)) = ^F,{t„t,)F, {t, , ) 

< f 2 < fi 

_,_g-W+«^)(f2-», 



2P'{P'-k')1 P 

\+p{t^-t,) 

2P'{P'-k')[ p' 



t,<t. 



.-^('.«^)||jJ/('=«^«'«')(£(g£(g7,.(r3)/,.(0) = ^F,(r,OF3(r,,g 

'^pK 



Thus, we have 

Ti = lim e-^"' 1 1 e^<'' {t„t,) dt,dt, 





\69al 
432^ 



2 /J4 



4/f'(;ff + /t-) 



T: ^ Ume-^^' f f /C'^'^'a U,L)dtdL = ^— 

rf ^Ume-^^' \e'^"'''-^D':'(t„t,)dt,dt, = ^ ' ^ ^ , 

^ lime-^^' f f e''(""^)D<'' U,tM,dt, = f^''^ , 
Furthermore, 

lime-'^' J j/C'*'^) {sl{t,)sl{t,))dt,dt, =T,+T, +r,<" +rf > (S40) 



Urn e-'^' I J e''*'' ' (if (fj (f^ )) dt.dt^ =T,+T^+ 4r,<" + 2r;" , i = l,2 (S4 1 ) 



r f 

hme-''"||e^<''"'^* l^s^{t,)s,{t^)s^{tj)dt,dt^ =T,+ r/" + 2r,<" , « = 1,2 (S42) 


r t 

lime-''" {s^(t,)s,{t,}s,(t^)s,{t,))dt,dt, = 7] +7^ +rf +?;<'' , / = 1,2 (S43) 



Note that 



^ _ (^11 "*"^^12 ^22) rp , (^11 ^22) rp J- 

<^2 ~ . -'1+ . -'2+^12-'3 + 



(^11+^,2) 



2 I 8l\S22 



2 



{8,2+822) + 



2 , ^11^22 



rp{2) , 811 {811 +^^12) ^(1) , ^22(^22"*"^^12) ^(2) 
■'4 + - ^5 + ^ ^5 



'(1) 
4 

(S44) 



In addition, note that 

= iSn+2s -822K limF.(M). limF3(r,0 



f->00 



or 



^ _ Ull + 2gl2 + ^22 ) , ^ll-^f + ^220-: 



8y3' APk{P + k) 



we can compute and obtain 



r - "° 



k{P + k) 



In particular, we have in the presence of extrinsic noise only 



R (0) = ^'^^ 



Ap" 



l, 3(gi+g2) , 9(g,+g,)^ 



7l(gll+2gi2+g22)'o-£ 



864y5' 



whereas in the presence of intrinsic noise only 



R (0)= 



C7„^ (2/?+^)(g'^'+g2^2) 



2pK{P + K) AK{P + Kfp' 

[ep' +9pK + 2K-')glalal [gn^^^ + Sii^^l)^ +{8^+822)' 



\1{P + 1k)(1P + k)k^{P + k\ p' 



\6K'{p + Kf p* 



(S45) 

(S46) 
(S47) 



(S48) 



(S49) 
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